Chris Lamb: Installation birthday
$ apt install installation-birthdayEnjoy, and patches welcome. :)
$ apt install installation-birthdayEnjoy, and patches welcome. :)
diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues.
strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build.
diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues.
strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build.
debootstrap
) rootfs tarballs.
The project is named Debuerreotype as an homage to the photography roots of the word snapshot and the daguerreotype process which was an early method of taking photographs. The essential goal is to create photographs of a minimal Debian rootfs, so the name seemed appropriate (even if it s a bit on the mouthful side).
The end-goal is to create and release Debian rootfs tarballs for a given point-in-time (especially for use in Docker) which should be fully reproducible, and thus improve confidence in the provenance of the Debian Docker base images.
For more information about reproducibility and why it matters, see reproducible-builds.org, which has more thorough explanations of the why and how and links to other important work such as the reproducible builds effort in Debian (for Debian package builds).
In order to verify that the tool actually works as intended, I ran builds against seven explicit architectures (amd64
, arm64
, armel
, armhf
, i386
, ppc64el
, s390x
) and eight explicit suites (oldstable
, stable
, testing
, unstable
, wheezy
, jessie
, stretch
, sid
).
I used a timestamp value of 2017-05-16T00:00:00Z
, and skipped combinations that don t exist (such as wheezy
on arm64
) or aren t supported anymore (such as wheezy
on s390x
). I ran the scripts repeatedly over several days, using diffoscope
to compare the results.
While doing said testing, I ran across #857803, and added a workaround. There s also a minor outstanding issue with wheezy
s reproducibility that I haven t had a chance to dig deep very deeply into yet (but it s pretty benign and Wheezy s LTS support window ends 2018-05-31, so I m not too stressed about it).
I ve also packaged the tool for Debian, and submitted it into the NEW queue, so hopefully the FTP Masters will look favorably upon this being a tool that s available to install from the Debian archive as well.
Anyhow, please give it a try, have fun, and as always, report bugs!
strip-nondeterminism
and also filed #862073 against dak to upload buildinfo files to external services.misc.git:has-only.py
, and started looking at Britney.ssh
or controlling light and music with an webbrowser without authentication (besides being in the right network).
(This wasn't the hackathon per-se, but some of us appreciated these sights and so we thought you would too.)
Many thanks to:
0.033-1
and -2
were uploaded to unstable by Chris Lamb. It included contributions from:0.6.1
and 0.6.2
were uploaded to unstable by Ximin Luo. It included contributions from:diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues.
strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build.
buildinfo.debian.net is my experiment into how to process, store and distribute .buildinfo files after the Debian archive software has processed them.
diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues.
buildinfo.debian.net is my experiment into how to process, store and distribute .buildinfo files after the Debian archive software has processed them.
strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build.
diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues.
buildinfo.debian.net is my experiment into how to process, store and distribute .buildinfo files after the Debian archive software has processed them.
[ Chris Lamb ]
* Optimisations:
- Avoid multiple iterations over archive by unpacking once for an ~8X
runtime optimisation.
- Avoid unnecessary splitting and interpolating for a ~20X optimisation
when writing --text output.
- Avoid expensive diff regex parsing until we need it, speeding up diff
parsing by 2X.
- Alias expensive Config() in diff parsing lookup for a 10% optimisation.
* Progress bar:
- Show filenames, ELF sections, etc. in progress bar.
- Emit JSON on the the status file descriptor output instead of a custom
format.
* Logging:
- Use more-Pythonic logging functions and output based on __name__, etc.
- Use Debian-style "I:", "D:" log level format modifier.
- Only print milliseconds in output, not microseconds.
- Print version in debug output so that saved debug outputs can standalone
as bug reports.
* Profiling:
- Also report the total number of method calls, not just the total time.
- Report on the total wall clock taken to execute diffoscope, including
cleanup.
* Tidying:
- Rename "NonExisting" -> "Missing".
- Entirely rework diffoscope.comparators module, splitting as many separate
concerns into a different utility package, tidying imports, etc.
- Split diffoscope.difference into diffoscope.diff, etc.
- Update file references in debian/copyright post module reorganisation.
- Many other cleanups, etc.
* Misc:
- Clarify comment regarding why we call python3(1) directly. Thanks to J r my
Bobbio <lunar@debian.org>.
- Raise a clearer error if trying to use --html-dir on a file.
- Fix --output-empty when files are identical and no outputs specified.
[ Reiner Herrmann ]
* Extend .apk recognition regex to also match zip archives (Closes: #849638)
[ Mattia Rizzolo ]
* Follow the rename of the Debian package "python-jsbeautifier" to
"jsbeautifier".
[ siamezzze ]
* Fixed no newline being classified as order-like difference.
reprotest development
reprotest 0.5 was uploaded to unstable by Chris Lamb. It included contributions from:
[ Ximin Luo ]
* Stop advertising variations that we're not actually varying.
That is: domain_host, shell, user_group.
* Fix auto-presets in the case of a file in the current directory.
* Allow disabling build-path variations. (Closes: #833284)
* Add a faketime variation, with NO_FAKE_STAT=1 to avoid messing with
various buildsystems. This is on by default; if it causes your builds
to mess up please do file a bug report.
* Add a --store-dir option to save artifacts.
Other contributions (not yet uploaded):
website.git notifications now go to
#reproducible-builds. Thanks to
danielsh and the KGB bot maintainers.arm64
architecture was fully tested in all three suites in just 15 days. Thanks again to Codethink.co.uk for their support!pg_dump
with -O --column-inserts
to make easier to import our main database dump into a non-PostgreSQL database. (mapreri)armhf
network: CPU frequency scaling was enabled for three
Firefly boards, enabling the CPUs to run at full speed. (vagrant)year | number of bugs closed |
2011 | 63 |
2012 | 28 |
2013 | 73 |
2014 | 5 |
2015 | 150 |
2016 | 95 |
diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues.
strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build.
trydiffoscope is a web-based version of the diffoscope in-depth and content-aware diff utility. Continued thanks to Bytemark for sponsoring the hardware.
buildinfo.debian.net is my experiment into how to process, store and distribute .buildinfo files after the Debian archive software has processed them.
Great pleasure is to be found not only in keeping up an old and established friendship but also in beginning and building up a new one. Reading this in a beautifully svelte hardback, I tackled a randomly-chosen letter per day rather than attempting to read it cover-to-cover. Breaking with a life-long tradition, I even decided to highlight sections in pen so I could return to them at ease. I hope it's not too hackneyed to claim I gained a lot from "building up" a relationship with this book. Alas, it is one of those books that is too easy to recommend given that it might make one appear wise and learned, but if you find yourself in a slump, either in life or in your reading habits, it certainly has my approval.
strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build.
jenkins.debian.net runs our comprehensive testing framework.
dpkg
1.18.14 has migrated to stretch.debdiff(1)
what patch(1)
is to diff(1)
. It is a
general tool that is not specific to Reproducible Builds. This week, Ximin Luo
worked on making it more "production-ready" and will soon submit it for
inclusion in devscripts.
reprotest development
Ximin Luo significantly improved reprotest, adding presets and auto-detection
of which preset to use. One can now run e.g. reprotest auto .
or reprotest
auto $pkg_$ver.dsc
instead of the long command lines that were needed before.
He also made it easier to set up build dependencies inside the virtual server
and made it possible to specify pre-build dependencies that reprotest itself
needs to set up the variations. Previously one had to manually edit the virtual
server to do that, which was not very usable to humans without an in-depth
knowledge of the building process.
These changes will be tested some more and then released in the near future as
reprotest 0.4
.
Next.